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ABSTRACT 

Legitimate external groups that seek accountability 
for educational outcomes are identified, and attention is directed to 
choices for designing an institutional assessment, as well as 
principles to guide institutions* response to accountability demands. 
Appropriate domains of accountability include: (1) the authority of 
state government to regulate and govern public colleges; (2) the 
review of institutions by regional accreditation bodies; and (3) 
calls for individual colleges and universities to provide information 
to guide "consumer choice." Areas in which colleges must choose how 
to proceed with an assessment include: what to assess, how to 
organize the assessment, and the manner and extent to which 
assessment results are communicated. Principles that institutions can 
follow in their response to accountability demands include: respond 
visibly to domains of legitimate external concern, build on strength, 
&Ad show action. Particular actions that institutions can undertake 
include: extending or initiating post-testing of basic skills, 
experimenting with new approaches to student evaluation within the 
regular curriculum, and making assessment a visible part of the 
curriculxxm and important administrative processes. (SW) 
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Assessment, Accounrabl I Ity and Improvement: 
Managing the Contradiction 



Assessing the results of undergraduate education continues to receive 
considerable attention across all sectors of higher education. Growing out of 
a wider movement of undargraduate reform, current Interest demonstrates 
Increasing recognition within the academy that assessment Is an Important tool 
for ongoing Instructional Improvement. Recently, however, a new set of 
assessment Issues has emergod. These Issues Involve the use of assessment as 
an accountaoM Ity device, to assure external bodies charged with governing and 
funding higher education that their Invested resources are well spent. How 
ought Institutions respond to these Issues? How can they adequately meet 
growing deniands for accountability and at the same time undertake serious 
Internal Investigations of Instructional effectiveness? 

In posing this question. It Is Important to recall that the current "assessment 
movement" In higher education has Its roots In two quite different traditions. 
Its first antecedent has a long history, and concerns the use of assessment as 
an Integral part of Instruction* This tradition Is embodied In many 
time-honored currlcular practices (among them the senior comprehensive 
examination) that are currently being "rediscovered" on many col lege campuses. 
It also embraces more recent and Innovative "assessment center" approaches* 
Early efforts at programmatic evaluatinn for Instructional Improvement — for 
example, the Examiner's Office at the University of Chicago and the General 
College at the University of Minnesota (Pace 1979)~are also a part of this 
approach- to assessment. Finally, the values and lessons of this tradition are 
written Into virtually every recent call for undergraduate reform (NIE 1984, 
AAC 1985, NEH 1984). 

A second tradition holds that the primary purpose of student assessment is to 
demonsYave the effectiveness of public education to a variety of external 
constituencies. Certainly no one will deny the current strength of this 
tradition* As documented by a recent survey, two-thirds of the states now 
claim or plan an assessment Initiative of some kind, and over QQ% of the fifty 
state higher education officials surveyed expect state activity In the realm of 
assessment policy to become more prominent (Boyer et al. 1987). But assessment 
for accountability too has strong historical roots. Though largely foreign to 
higher education, a primary reason for large-scale testing of elementary and 
secondary s<:hool students since the turn of the century has been to reassure a 
variety of <^xternai publics (Resnlck 1982). Indeed, as Resnick argues, the 
tenor of assessment debates Is familiar even In their earliest manifestations. 
Large-scale cognitive testing programs are particularly resonant with an 
American public character that demands demonstrable results for resources 
Invested* They are also compel llngly consistent with a "rational*' conception 
of public policy that emphasizes Information-based decisionmaking for purposes 
of incremental improvement. Furthermore, testing for these purposes has 
certainly not been novel for postsecondary audiences in such areas as teacher 
education and In the certification process for professional practice in a wide 
range of fields. 

Though botn these traditions have a long history, their contraalction poses for 
most Institutions a considerable dilemma — a fact that academic administrators 
have been quick to recognize (EI-Khawas 1986). If Instructional improvement is 




to be seriously attempted, most administrators recognize that the process will 
take considerable time. Most also realize that serious efforts wil I require 
approaches to assessment that go far beyond standardized tests. As precarious 
organizational leaders, moreover, academic administrators understand that 
Implementing meaningful campus change demands persuasion of a suspicious, 
reluctant, and generally powerful faculty. If, on the other hand, demands for 
public accountability are to be adequately met, the kinds of Information 
provided must be external ly credible, must be capable jf supporting comparative 
judgments on relative institutional and program performance, and must be 
straightforward enough to be comprehensible to a lay audience. To convince a 
doubting legislature or board that the Institution and Its faculty are serious 
ab'-ut assessment also requires academic administrators to design a process that 
Is organizationally prominent and that Is capable of producing quick results. 
Nor Is this balancing act made easier by the form of emerging state 
Initiatives. Most such Initiatives are sincere In their claim that, like 
Institutional assessment, their primary purpose is local instructional 
improvement. But coincidence of motives is not sufficient to overcome 
legitimate def enslveness on the part of institutions; for them, the source of 
such "encouragement" alone raises many potential problems. 

Given the terms of this contradiction, how can and should institutions respond? 
The purpose of this paper is to provide some initial answers. Addressing the 
Issue adequately, however, requires accomplishing several tasks. First, It Is 
by no means clear that "external demands for accountabil Ity" are of th€. same 
kind, and different demands may require different kinds of response. Several : 
distinct areas of external accountability must be distinguished and the 
assessment domains appropriate to each established. Secondly, general 
contradictions between accountability and Improvement are ecribodled In a set of 
concrete choices about how to design and Implement a local assessment effort. 
Among the choices that mus\ be made are what particular dimensions of student 
performance to assess, the kinds (and particularly the degree of 
standardization) of Instruments and assessment procedures to adopt, where to 
locate responsibility for assessment In the Institution's organizational 
structure, and how to use and communicate assessment results. Final ly, 
discussion of these Issues suggests some principles for action. Although these 
principles are heuristic, they suggest a number of concrete, short-term 
strategies for Institutions to consider. 



Some Appropriate Lkxnalns of Accountabll Ity: Who Owes What to Whom? 



External pressure to account for educational results has come from many 
sources. Most visible, of course, are the recent actions of state government, 
either directly through legislative statute or Indirectly through higher 
education coordinating and governing boards. Less visible but nevertheless 
Impo»-tant is heightened Interest In assessment on the part of regional and 
professional accreditation bodies. Least focused but perhaps most pervasive 
have been a variety of calls for "consumer protection" (for example, Bennett 
1985) • Each of these sources represents a somewhat different kind of 
accountabi I ity. 

But what exactly is meant by "accountability"? Recently, observers In higher 
education have argued that the concept has two distinct d i mens i ons — d I scharg I ng 
assigned institutional missions effectively and demonstrating that these 
responsibilities have In fact been effectively discharged (Jones and Swell 
1987) • Distinguishing between the perforrance and communications aspects of 
accountability Is Important because these two requirements may differ 
substantially for different constituents. Furthermore, specifying the 
particular dimensions of accountability with respect to performance and 
communication requires a careful analysis of both the Information needs and the 
assigned responsibilities of each constituency to whom accountability Is owed. 
In each case, the analysis must determine: a) the basis upon which 
accountabll ity is owed; b) the substance of the obi Igatlon with respect to 
specific types of performance; and c) the kinds of Information about 
performance appropriate to provide evidence of effective performance. 

Given this requirement, appropriate domains of accountabll ity for partlci'lar 
external constituencies are both limited and specifiable. And given the fact 
that Institutions are likely to see in ^ox external request for information a 
substantial threat. It is important that they recognize which of these are at 
least arguably legitimate, 

1, State Government. For reasons of substantial investment ano established 
governance, state authorities can legitimately claim from public colleges 
and universities considerable accountability for results. Given the 
staters charge to maintain and develop Its higher education asset as a 
whole, moreover, private institutions may also be appropriately subject to 
limited accountability demands. These are reinforced where state-funded 
financial aid programs allow citizens of the state to attend private 
Institutions at their discretion. 

Historically, state governments have funded, regulated, and governed higher 
education In pursuit of three basic goals (Ewel I 1985), A first Is access. 
Part of the justification for equal access Is philosophic: If college and 
university Instruction does In fact provide opportunities for individual 
economic and social mobility, discriminatory access to these benefits is 
disallowed. But part of the justification for access Is also instrumental: 
given the need for trained manpower for economic development, states cannot 
afford to w?^ste potential talent. Historically, accountability for access 
has been limited to inputs. Most states, for example, currently require 
public Institutions to report both current enrollments and new students by 
ethnicity, gender, and (more rarely) categories of economic or educational 



disadvantage. Ip a few cases, primarily those southeastern states affected 
by federal desegregation Initiatives, access Is also treated In limited 
output terms. Institutions are required to monitor and report persistence 
and degree completion rates for Identified ethnic subpopulatlons. 

A second historic foundation of state Involvement In higher education Is to 
sustain economic development. Applied research and development capacity 
within public colleges and universities, and the provision of trained 
manpower to sustain growth in Identified critical occupational areas are 
key arenas of concern in pursuit of this goal. Here accountability resides 
primarily In a periodic Inventory of assets. University research capacity 
Is documented in the form of the availability and adequacy of equipment, 
funding, and personnel. The provision of trained manpower Is monitored In 
terms of the capacity, distribution, and quality of available professional 
programs In Identified areas. But some output dimensions are also 
occasionally included- Research effectiveness, for example. Is sometimes 
assessed In terms of success in garner? - federal funds, the award of whicn 
Is generally based on peer review. Sim..jrly, the provision of trained 
manpower is sometimes judged by the production of graduates in particular 
f iel ds. 

A third reason for state involvement is to ensure that citizens of the 
state possess minimum levels of skills In critical areas of "functional 
citizenship". This claim Is partially bound up with concerns about access 
and economic development: to sustain development, the state needs to count 
on £JLL its citizens possessing critical general skills* The claim Is also 
Justified In terms of the responsibilities of citizen participation and 
quality of life. Though publicly acknowledged, this arena is In most 
states not a part of established accountability reporting. States such as 
New Jersey, Georgia, and Florida that require basic or collegiate skills 
testing of public college and university students constitute significant 
exceptions. 

State authority is also ultimately responsible for maintenance of the 
higher education asset as a whole. Here responsibility Is assumed not for 
a particular domain of performance, but for sustaining an educational 
capacity over time. Traditionally, maintenance of the asset Is approached 
in resource terms — for example, the adequacy of finance, of physical plant 
and equipment, of faculty, and of currlcular and program offerings. The 
notion of capacity also Includes the presence of articular management and 
administrative processes deemed Importani to sustain Institutional 
viability. Among these are financial accounting and resource allocation, 
personnel review and recruitment, and Institutional planning and program 
eval uatlon. 

What do these four traditional, and largely legitimate, areas of state 
responsibility suggest as appropriate arenas of concern wlrh respect to 
educational outcomes? Broadly conceived, they argue for two distinct kinds 
of accountability. First, in areas In which the state has a legitimate and 
direct responsibility. Institutions may approDriafely be required to 
provide information on actual performance. Examples include the following: 

• Persistence and program completion rates for particular demographic 
and economic/educationally disadvantaged populations 



• Graduation, Job placement, and certif ication/pGrformance rates in 
selected occupations or professions deemed Important to attaining 
state economic development goals 

• Mastery of minimal basic skills In reading, writing and computation 
and evaluations of the effectiveness of remediation. 

In all such cases It may be legitimate for slates to collect such 
Information In standardized form and on a comparative basis. Beyond these 
areas, however, standardized comparative data collection will be generally 
Inappropr late^ 

Secondly, given the state's legitimate responsibility for maintaining the 
higher education asset. Institutions may appropriately be held accountable 
for particular processes or functions assumed to contribute to 
Institutional effect I veness. Among these are: 

• Mission review and analysis. Including specification of the kinds of 
Instructional outcomes Intended 

• Evaluation of Lae effectiveness of general education programs 

• Evaluation of the outcomes of Individual degree programs (perhaps 
under the rubric of program review) 

• Evaluation of student satisfaction and Individual goal attainment 

• Utilization of assessment results In Institutional decisionmaking 
processes. 

It Is consistent with state responsibility to require Institutions to 
present evidence that they are engaged In each of these processes In a 
meaningful way. It Is less consistent with this responsibility that they 
be explicitly required to report the resuUs of such processes. And It is 
not at all consistent with state responslbll ll^y that such evidence be 
reported In a standardized, comparative manner. 

Accreditation. Regional accreditation bodies have traditionally reviewed 
Institutions according to broadly defined criteria of viability. In most 
casesn viability has been defined In terms of assets and Inputs; only 
recently have outcomes been stressed. Professional accreditation bodies, 
in contrast, examine particular programs from the limited perspective of an 
occupation or profession. Here outcomes assessment may be required, but 
generally In the form of Individual certification to practice. 

The legitimate domain of professional accreditation has been much debated, 
and several national reports have cal led on professional accrediting bodies 
to provide greater choice and flexibility In curricula than current 
standards allow (NIE 1984). But there has been relatively little debate 
about the legitimacy of requiring demonstrations of professional competence 
on the part cf program graduates. Because professional accreditation 
signifies accountability to a particular professional community (and 
indirectly to the publics and clients served by that community), because 
the outcomes of professional education are comparatively easy to define and 



assess, ann finsMy Decause of the tamil rarity of assessment processes as 
an Ingredient of professional certification, there Is generally little 
debate over the legitimacy of outcomes assessment as a part of the 
accreditation process. 

Regional accreditation presents a quite different picture* Here 
accountability Is owed not to an external authority, but rather to the 
academy Itself. In extending accreditation to an institution, the academy 
as a whole recognizes and certifies that the institution meets certain 
minimum standards— as embodied, for example. In the willingness of other 
Institutions to accept Its academic credits in transfer. Also In this 
regard, the academy acts to protect "consumer Interest"; prospective 
students must bo assured that sufficient resources are available to enable 
them to complete their programs without substantial disruption. What are 
the legitimate boundaries of such accountabi Nty with respect to 
assessment? Answering this question means returning to some original 
grounding assumptions of accreditation, eittbodled In the principles of 
sel f-study, 

A first principle of self-study Is that It Is sel f- stuQy> That Is, the 
Institution undertakes an evaluation of Its operations and attainments in 
terms of Its own publicly expressed Instructional goals. This principle 
reflects the legitimacy of diversity. Because colleges and universities 
are founded for many purposes, serve many different types of students, and 
incorporate a range of different educational values and technologies, 
self-study allows Investigations of ef fecr I veness to proceed without 
Imposing a common, and presumably narrowing, set of standards, 

A second principle of self-study, howeve. , is that It Is sel f -study . That 
Is, It Is a process of Inquiry which, like other processes of academic 
Inquiry, Is subject to the canons of good research. Here accountability 
resides as much In method as In the result obtained. The peer review 
process Is often described In such terms— -a primary charge of visiting 
teams being to "validate" the self-study undertaken by each Institution. 

A final principle of self-study Is Improvement. Investigation Is not 
undertaken for Its own sake but rather to enable the insrltutlon to build 
on strengths and address Identified weaknesses. As a result, for 
accreditation a final domain of accountability Is the action taken. 
Institutions found wanting in particular areas, for example, are required 
to prepare periodic progress reports or to receive focused team visits. 

Viewed as elements of Institutional accountability, these three principles 
reflect the values of the acadeny with respect to any kind of Inquiry: 
appropriate inquiry Is generally held to be a) freely determined by 
individual scholars and value-neutral, b) methodologically rigorous, and c) 
ongoing and continuously refined over time. What do they particularly 
require In the area of assessment? At minimum, evidence of three kinds is 
needed: 

♦ Clear statements of Instructional Intent, including the kinds of 

instructional outcomes that the instlturion ano Its programs seek to 
produce 



• Documertarion of The processes Dy which the institution ensures that 
these goals are In fact being attained, together with appropriate 
evidence of attainment 

• Evidence of how the results of these processes are actual ly used by 
the Institutions to make Imprc emenxs. 

In each area, accountability Is focused primarily and appropriately on 
process. In no case Is assessment evidence to be separated from the 
institution's particular context or its unique Instructional mission. 
Moreover, what results are communicated as part of the process need not be 
directly comparable to those reported by other institutions. 

"Consumer Protection", By far the most diffuse area of accountability tor 
educational results are recent calls for Individual colleges and 
universities to provide Information to guide "consumer choice". Although 
gsnerally expressed by state and federal authorities, accountability here 
Is owed to a wide range of particular Individual students and parents faced 
with a choice of which of many available Institutions to attend. 

What should accountability of this kind legitimately encompass? A first 
answer starts with the claims to effectiveness that colleges and 
universities themselves assert. This answer — a demand for "truth in 
advertlslng"~asks institutions to carefully examine what they publicly 
claim to accomplish, and to substantiate such claims with reasonable 
evidence. The primary point of departure for accountability of this kind 
Is thus the Information that the institution provides to prospective 
consumers. Such Information Is typically located in many places. Among 
the most common are college catalogues, "view bocks", and other recruitment 
materl&is. Claims most often made through such media Include: a) 
successful preparation for and placement In favorable employment situations 
or graduate programs; b) a range of general knowledge and skills deemed 
Important for social functioning and individual fulfillment; and c) a 
distinctive set of personal values cultivated by the institution's history 
and traditions. Often these are accompanied by statements about the 
distinctiveness of the learning environment that the Institution 
provides — for example, references to small classes, opportunities for 
frequent contact with faculty, or claims about the use of particular kinds 
of i nstr uct 1 ona I techno I ogy . 

A second accountability demand !s more specific. In addition to the public 
claims made by colleges, potential consumers may have a legitimate Interest 
in particular areas of Instructional performance. Recent legislation on 
community colleges in California, for example, requires each district to 
report periodic "consumer" performance to potential students on a variety 
of dimensions (California General Assembly 1986). Among many such claims. 
It Is possible to Identify at least three minimal domains of legitimate 
"consumer" accountability: 

• Potential students of different background and ability have a 
legitimate claim to know +helr chances of actually completing a 
particular program of study. Furthermore, they should be aware of 
how long the process typical ly takes. Both are prudent questions 
for any potential Investor. 



• Pofenti'ai students ciso have a legitimate right to know the 
institution's actual record In placing students In relevant 
employment situations or In graduate training. Once again, to guide 
choice, such Information should Indicate any relevant differences in 
success experienced by different types of students, 

• Finally, potential students have a claim to Information about the 
past success of the Institution In producing "satisfied custotners,'* 
While to sane extent expressed satisfaction will be embodied in the 
prior two dimensions, a large component of satisfaction is embodied 
in personal goal attainment. 

Most Institutions already can provide a great deal of information on each 
of these dimensions (Lenning and Cooper 1978) • And certainly institutions 
need provide no guarantees of individual success. Potential students, like 
other kinds of service consumers, must recognize that the investment they 
are making requires that they keep their part of the bargain by actively 
engaging in the educational process. 
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Accountability vs. Improvement: Three Arenas of Choice 



Regardless of source, external demands for accountability will severely 
constrain an institution's ability to develop local assessment mechanisms that 
meet their own needs. Establishing legftlmate domains of external 
accountability with respect to instructional outcomes can mitigate these 
constraints. But It cannot remove them. Institutions will be left with 
difficult choices about how to proceed with a particular assessment. In what 
specific areas must such choices be made? Emerging experience suggests three 
main arenas. 

What and How to Assess , Probably the most basic set of choices that any 
Institution must make concerns the kinds of outcomes to be Investigated and 
the types of methods to be employed. These choices constitute Initial 
answers to the questions of what and how to assess. Prior to answering 
both, however. Is the question of Jibx the effort is to be undertaken (Ewell 
1987). if Its primary purpose Is "formative"— as suggested by most 
emerging literature on assessment — both the particular outcomes to be 
assessed and the methods used in doing so may be multiple and creative. If 
the primary purpose Is "summatlve" — as suggested by many demands for 
accountabi I Ity— answers may be quite different. 

A first Issue concerns the coverage of assessment: what exactly Is to be 
examined? Potential outcomes of the col lege experience are many and 
include dozens of potentially "assessable" attributes. No Institution can 
hope-to assess them ail. Which should be chosen for explicit 
investigation? For local Improvement, the most obvious choices are those 
most closely related to institutional purpose or those that most clearly 
reflect Institutional character or distinctiveness. Left to themselves, 
therefore, the outcomes dimensions for a public community college and a 
private liberal arts college may markedly differ. The first will 
appropriately emphasize basic skills and success In entry-level employment; 
if attitudes are tapped these will m^st appropriately be limited to student 
satisfaction. For the small private liberal arts college the answer will 
be quite different — particularly If the Institution emphasizes a particular 
values orientation or instructional philosophy. Here priority may 
appropriately be given to assessing student value development and the 
mastery of generic higher-order intellectual skills. 

External demands for assessment constitute an ^ priori threat to an 
Institution's choice of appropriate outcomes dimensions. First priority In 
assessment will Inevitably be given to those areas perceived as important 
for general accountability, regardless of their correspondence to a given 
Institution's purpose. Certainly some outcomes dimensions are arguably of 
sufficient general Importance that all Institutions can legitimately be 
called upon to assess them. The dange- comes when such dimensions are not 
well specified by external authorities, or when lists of "suggested" 
dimensions are provided. For example, Colorado's House Bill 1187 notes 
that as a part of accountability Institutions might report information on 
"knowledge, intellectual capacity, and skills, and many Include other 
dimensions of student growth such as self-confidence, persistence, 
leadership, empathy, social responsibilities, understanding of cultural and 
intellectual differences, emoloyabi i Ity, and transferrabi I ity." Similar 



lists characterize much emerging accountaDil Ity legislation. The problem 
In these cases Is that Institutions wll! see such suggestions as 
requirements, and will consequently downplay or abandon far more 
ap;)roprIate local lines of Inquiry. 

A second difficulty Is similar. Many colleges and universities are 
currently under mandates to design and Implement assessment within a very 
short period; In some states, Missouri and Virginia for example, 
Institutions have been given less than a year to make basic assessment 
choices. Growing pressure for reporting results on a short time frame may 
Induce institutions to investigate only those outcomes dimensions that are 
relatively easy to assess. One manifestation of this difficulty is a 
growing tendency for institutions to quickly and uncf Itlcally adopt an 
aval I able commercial assessment Instrument. This choice is compelling 
because It obviates the need for a time-consuming local process of 
Iiistrument development* Uncritically adopting such an Instrument without 
careful examination, however, constitutes a virtual surrender of the 
initiative about what to assess. 

This growing phenomenon also Illustrates a third difficulty. Many 
Institutions feel under considerable pressure to choose a commercial 
standardized instrument because such instruments are perceived to be more 
externally "credible" than locally designed alternatives. Partly this 
perception is due to the nature of standardized Ins+ruments--partlcular I y 
their ability to provide deceptively concrete performance Information In 
quantitative (and therefore comparative) terms. Partly It Is due to the 
I nst'^ jment's source, as locally designed assessment procedures will always 
be cpen to potential charges of bias. Given both tendencies, the products 
of a commercial vendor look particularly attractive In meeting external 
mandates. 

Organizing Assessment, A second arena of choice In designing an 
Institutional assessment effort is how best to organize the activity. 
Dimensions of choice in this regard include whether or not to assign a 
par:'':ular office or individual with prima responsibility for coordinating 
assessment ef forrs, where in the administrative structure to locate this 
responsibility, and the integration of assessment efforts with ongoing 
institutional planning and decisionmaking. Here again, external 
accountability demands can considerably constrain institutional choice. 

One problem is the likelihood that demands for external reporting will lead 
to a more centralized assessment effort than are appropriate for an 
Instructional improvement effort. Institutional experience with assessment 
strongly suggests that the process is at its best for improvement when it 
is highly decentralized (Ewel I 1984, Banta 1986). Considerable diversity 
among instructional objectives and learning environments is best 
accommodated by letting acaoemic units experiment with assessment on their 
own terms. Not only are the resulting activities more congruent with the 
program's instructional objectives, but the "feedback loop" between 
information and action Is short. Information generated at the departmental 
level can be quickly and efficiently utilized by proctlclng faculty in 
actual classr-oom settings. 
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txternal ©ccountabi I Ity promotes an opposite organisational Imperative. If 
assessment information Is Intended primarily for dn external audience 
cenrrally organized data col le'^Mon both gains efficiency and ensures 
consistency In the information reported. At the same time, central Izatlon 
provides additional aids to bureaucratic reporting. External authorities 
can be assured that explicit assignment of outcomes-reporting 
responsibilities to a particular office mecns that they will consistently 
know where to turn for such Information, and with whom to negotiate. 

Certainly, an assignment of clear responsibility *or assessment can 
considerably enhauce local Implementation— -regardless of whether the 
primary purpose is Ir.rovement or accountabi : Ity. Indeed, coordinating 
large-scale data collection procedures virtually requires a central Ized 
function. Centra! office? can also serve as reservoirs of scarce technical 
expertise with respect to testing and measurement, and In the 
Interpretation of assessment results. Moreover, some Important assessment 
domalns~partlcularly those associated with general education—cannot be 
addressed piecemeal. Explicit and v'slble assignment of responsibility for 
assessment to a particular office or Individual has also helped campuses to 
symbolize their ongoing commitment to Instructional evaluatlc; and 
Improvement, The success of such oft Ices as the Learning Research Center 
at the University of Tennessee, KnoxvM le, and of the Instructional 
Evaluation Office at Alverno College provide considerable evidence of these 
benefits. Problems arise only when the primary reasons for cenrral Izatlon 
are associated with requirements for external reporting. 

A second croanlzatlonal Issue concerns the linkage ber^een those 
responsible for assessment and actual academic practice. In this case more 
subtle organizational problems are apparent. Compliance demands may Induce 
Institutions to create assessment efforts that are recognizably dhstinct 
from other domains of Institutional activity. For example, several states 
now require Inst It' Mons to submit "Institutional Assessment Plans" that 
describe local procedures for gathering Information and using the results 
(Beyer et al. 1987). Such mechanisms are admirable In their recognition 
that In;5tItutions may appropriately differ In their approach* But they 
carry a strong Implicit message that assessment should be organized as a 
distinct activity. Experience with effective assessment practice, on the 
other hand, suggests that Institutions should Integrate assessment with 
other ongoing academic plarnlng and evaluation activities. Often the 
appropriate vehicle for such Integration Is an existing Institutional 
process of budget planning or program review. But "burying" assessment 
activities In such existing mechanisms may convey to external authorities 
lack of seriousness about Implanentlng assessment. Faced with a choice, 
institutions nay increasingly opt for a distinctive and visible effort, 
regardless of Its internal organizational consequences. 

One manifestation of this choice has been an Increasing tendency "^o assign 
responsibility for assessment to an Office of Institutional Research. 
While this assignment reflects recognition that IR offices are often well 
equipped to analyze. Interpret and disseminate Information, It also subtly 
dlf ferentietes the assc^ssment function from the domain of academic affairs. 
Linkages between Offices of Institutional Research and departmental faculty 
are generally weak, and most faculty see Institutional Research as an 
administrative actfv^ity. More Import.^ntly, the center of gravity of 
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i fiSt I "tu't I CmG I rssosrch hss in recsrit years Inci 6asiMcjiy bnifted towdrd 
extornal accountability. As a result, assigning responsibility for 
assessment to an IR office sends an unmistakable Internal signal that the 
primary purpose of the exercise Is to satisfy the demands of external 
const lt*'encles. 

3. Communicating Ag^essment Results , A third arena of choice concerns the 
manner and extent to which the results of assessmen*^ are to be 
communicated, and It Is here that the 'St basic contradictions between 
accountability and Improvement arise, t^th functions require effective 
communication, but the kinds of communication most appropriate to each 
setting are markedly different. Certainly communication Is a critical 
Ingredient of accountability. Regardless of Its domain, the concept of 
accountability requires not only performance Itself, but also require*^ 
public presentation of evidence that requisite performance has been 
achieved. Accountability Information must also be of a particular 
kind — generic, summetlve, and easily comprehended L/ lay audiences. The 
use of Information for Improvement requires a quite different kind of 
communication. As documented by considerable research on the 
organizational utilization of evaluation Information, the most useful 
processes of communication are problem-driven, tailored to the particular 
needs of the user, and iterative (Braskamp and Brown 1980). Indeed, the 
best util Izatlon processes appear to be those In which data gatherers and 
program administrator's are able to engage In continuous dialogue on the 
Interpretation of evidence. Emerging experience atout the Institutional 
use of assessment results seems to confirm this more general finding 
(Klnnlck 1985). 

This basic conlradlction is embodied In several concrete Issues. First, 
the public nature of accountability, and the fact that most accountability 
Information is statistical, means that considerable attention will be given 
to the value of a given statistic rather than to the phenomenon that I les 
behind It. This Is a familiar difficulty In public policy. In higher 
education we have seen numerous examples of an Institution's ability to 
influence such "performance Indicators" as program cost without making 
fundamental changes In what It Is real ly doing. The tendency to focus on 
the indicator Itself is particularly strong when comparative data are 
presented. Probably the most pervasive (and legitimate) fear of public 
institutions with respect to mandated assessment is that the process will 
lead to uninformed judgments on relative "quality" that do not take Into 
account important dlffe'-ences In Institutional missions, resources, and 
primary cllenleies. Under such clrcumstancer , the pressure on Institutions 
to withhold or distort performance information can become considerable. 

A second difficulty Is related, and concerns the particular problems 
associated with communicating negative Information. On the one hand, 
experience indicates that sensitive use of negative Information constitutes 
a powerful lever for Internal change. Indeed a classic case of successful 
Information use is Northeast Missouri State University's use of data 
documenting regression In student mathematical ability to address and 
rectify serious curricular deficiencies (McClaln 1984). Seme institutions, 
moreover, have been unable to mobilize significant internal interest In 
reforming general education because results of the ACT-COMP examination 
revealed student performance to be above national norms. Both situations 
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reveal the power of a negative finding to creatively focus Institutional 
attention and to mobil Ize action. 

These Incentives are reversed when negative evidence must be communicated 
to external constituencies. Partly this is because of the public nature of 
the exercise. As several Institutions have recently discovered to their 
detriment, negative assessment findings make good copy for local media. 
But there Is also a strong fear on the part of institutions that negative 
evidence of performance will be used to "punioh" Institutions by 
withholding resources or by eliminating prograns. Such fears have long 
been a part of statewide program approval and review processes and have 
subjec+eo the Information use^j In such processes to considerable pressure. 
Overcoming such fears has required considerable time, and demands repeated 
demonstration by state authorities that they are not simply looking for new 
ways to reduce budgets. 

A final communications difficulty concerns the general lack of articulation 
between requirements for communicating assessment results and existing 
accountability requirements that mandate complex data collection and 
reporting. Many states raquire assessment to be added to existing 
accountability requirements in such areas as mission review, program review 
and approval, and enrollment/financial reporting. In California, for 
example, the outcomes-reporting provisions established In recently passed 
accountability legislation for the state's community colleges are added to 
an already complex set of reporting requirements. In Colorado, 
accoui.tabl I Ity reporting and Its associated resource consequences are but 
one of five parts of a general bill that establishes mandated admissions 
requirements for all public Institutions, procedures for program 
discontinuance, and methods for disbursing "centers of excellence" funds. 
In such cases there Is generally little effort to articulate assessment 
reporting with many other related requirements for commun f eating 
Information on Institutional condition and performance. By Increasing the 
complexity of reporting, this situation adds to an administrative bui den 
that Is already large. Moreover, by divorcing the communication of 
assessment results from ether proces:;83 related to the extent and condition 
of academic programs. It reinforces tendencies to divorce assessment from 
Institutional decisionmaking. 



Some Principles of Institutional Response 



With major contradictions between accountability and improvement identified, it 
Is possible to describe some promising avenues for institutional response. 
Here It Is Important to remember that the rhetoric of external calls for 
assessment visibly stresses the priority of local Improvement, This means that 
contradictions are often more apparent than real, as they arise as they are 
based upon what institutions lliinJi are the motives and needs of external bodies 
than a full understanding of what these constituencies actually Intend. In 
some areas institutional and state concerns really are coincident. Such casc^. 
If they can be identified, should constitute particular priorities for 
institutional action. Certainly effective Institutional response will require 
administrators to take the initiative across a range of such "win/win" 
situations. But how can these situations best be identified? Before 
presenting some examples of promising practice. It Is useful to note some broad 
principles of response. 

1* Respond Vtstbiy to Domains r>f Leg Iti mat e External Hnncern , Hot all demands 
for information on Instructional performance are illegitimate. In many 
cases, they arise out of real needs to discharge Important 
responsibilities. For state authorities, discharging such responsibilities 
may benefit all of higher education. Ao the cases of New Jersey, Tennessee 
and Florida attest, evidence of need or effectiveness provided by both the 
existence and the results of state assessment prograns can ground powerful 
arguments for Increased public support for hlyner education. Responding 
visibly and In good faith to emerging accrediting initiatives with respect 
to educational outcomes can similarly help restore the present low estate 
of voluntary accreditation as a credible process of Improvement and quality 
assurance. Visibly responding to Identified areas of statewide concern 
benefits can benefit Individual Institutions as well. One clear benefit is 
communication. By responding In good faith to external requests for 
information In such areas as the mastery of basic skills and In the 
placement of graduates In key manpower areas. Institutions can communicate 
their recognition that the needs of external authorities are In many cases 
legitimate. !n doing so, they have a much better chance of being heard 
when they argue that in other arenas the collection and use of assessment 
information Is best left In Institutional hands. At the same time, many of 
these legitimate demands for Information are relatively essy to satisfy. 
Information jn retention, program completion, and basic skills mastery do 
not present major measurement difficulties and are probably already on hand 
at most Institutions. Promptly complying with requests for Inf oj^mal ion in 
such areas will pay considerably more long-term dividends to InstltutI s 
than wll I responses based primarily on an attitude that any external demand 
for performance information ought to be resisted. 

2. Build Cn Strength . Most institutions already collect considerable 

Information on student performance. In many cases, the primary chal lenge 
in creating an assessmen-! urogram Is to identify. Inventory, and Integrate 
the results of many ongoing but uncoordinated data collection efforts. In 
responding to external demands for Information, this process is equally 
Important, not only because It results In efficiency but also because It 
saves time. Existing data bases and data col lection points should be 
carefully examined to determine the degree to which they can, with I Ittle 
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modification, provide the kinds of informarion required for external 
reporting. As above. It may be far more Important to show good faith by 
quickly communicating Information of Interest to external authorities, than 
to embark on a complex and costly effort to develop new data collection 
mechan Isms« 

3*, Show Act I on > Much of the recent concern about higher education's 

performance Is due more to a loss of confidence than to a desire on the 
part of external authorities to actively run the enterprise* As evidenced 
by emerging state programs, external authorities are far more Interested In 
fostering local se I f- Improvement than they are In Imposing an inappropriate 
conformity. If this trend continues. It may be Important for InstI t»;t Ions 
to communicate the actions that they have taken to adflress Identified 
strengths and weaknessas than simply to communicate the results of 
assessment* Again an emerging body of Institutional experience reinforces 
this conclusion. At public Institutions that have experimented with using 
Information about student performance In the budgeting or program review 
process (such as SUNY-Albany and the University of Tennessee, Knoxvllle), 
assessment Information Is always a beginning point for a proposed 
departmental course of action — presented together with its expected 
time-line and Its associated resource consequences* Responses such as 
these demonstrate not only accountability but also responsibility* The 
more Institutions can provide them, the more likely it will be that 
external authorities will allow Increased latitude In tailoring assessment 
methods and approaches to Institutional needs* If, on the other hand, the 
Institution's response Is to treat accountability merely as communicating a 
required set of statistics, the reaction of external authorities Is likely 
to be to Increase the number of things to be reported. 

Given thefe principles, what particular kinds of action can institutions 
undertake? A number of promising lines of adlvlty that combine two or 
more of these principles can be Immediately Identified. 

- Extend or Initiate oost-te sting of basic ski Ms , Most institutions 
currently engage In considerable pre-testing of student abilities in 
such areas as reading, writing, and computation. Their primary 
purpose Is to place students appropriately — either in a particular 
college-level sequence of courses, or In a remediation program 
designed to address Identified weaknesses. Externally designed 
stantlardlzed Instruments are general ly used In such processes 
although. In computation particularly, many Institutions construct 
their own. Only In rare cases, however, are students typically 
re-tested using the same (or any) Instruments to ascertain the 
competency achieved, or to assess the effectiveness of remediation. 
Initiating a local orogram of this kind can have considerable 
external payoff. . first responds proactlvely to an area of 
particular (and legitimate) external concern. Secondly, It ensures 
and demonstrates the effectiveness of the remediation function as a 
point of access. Final ly. It demonstrates the Institution's 
willingness to use standardized assessment Instruments where these 
are believed to be warranted. At least as Importantly, proposing and 
Implenentl ng a program of this kind will meet far less Internal 
resistance than will other alternatives* Many faculty will see such 
an Initiative as a guarantee that students possess Important 
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college-level skills before entering their own classrooms. More 
Importantly, few faculty Interests will be directly threatened by 
evaluating remediation. Finally, the technology of assessment need 
not be an Issue. The value of assessment in providing useful 
Infornation will already have been demonstrated, and most decisions 
about which Instruments to use and how to Interpret the results wll I 
already have been made. 

Experiment with new approach es to student evaluation w^tM n t h9 
regular curriculum. Because eroded confidence In college and 
university grading practices Is a major stHulus for external 
asressment, ar Institution's own grading system Is an obvious point 
of aitack for deveUpIng a credible approach to assessment. What can 
be done to restore lost confidence? Two kinds of activities, once 
common, are rapidly becoming important features of undergraduate 
curricula. The first are senior comprohensi ve examinations requiring 
a student to Integrate knowledge and ski I Is taught throughout the 
curriculum, in most cases, such examinations cover the student's 
major field, but ir. some they also require df^monstration of a range 
of general knowledge and skills. In some cases as well, senior 
comprehenslves are team-graded, and the multiple judgments of 
performance that the process Involves provide additional evidence of 
its credibility. The external Impact of such examinations is further 
Increased when they are aeveloped cooperatively among several 
departments or Institutions. In Tennessee, for example, a number or 
Institutions have cooperated In developing major field examinations 
as required by the Tennessee Higher Education Commission. A second 
emerging practice — using external examiners — further extends this 
logic. Drawn from faculty of equivalent departments In matched peer 
Institutions, external examiners can give useful feedback on the 
curriculum as a whole, as well as assessing individual student 
performance. At the same time, t^ey provide a "disinterested" check 
on local faculty grading practice. A current AAC )roJect using 
external examiners Is currently underway and the approach shows great 
promise In simultaneously enhancing curricular improvement and In 
demonstrating credible assessment practice. 

Seek the testimony of "satisfied customers". Graduates and former 
students are the ultimate products of an institution's Instruction. 
As such, their testimony can be particularly valuable In the 
political process of accountability. Because they are tne direct 
recipients and beneficiaries of instruction, their opinions of its 
effectiveness are by definition relevant. Because they are 
Interested parties, external agencies cannot politically afford to 
Ignore them. And because they are no longer associated with the 
institution and can be presumed to be actively testing the knowledge 
and skills that they acquired In the marketplace and In further 
education, their responses are generally credible. Furthermore, 
collecting Information from former students Is not technically 
daunting. Most of the required Information can be ootained by 
survey^ and many techniques are available for ensuring reasonable 
response rates. Most colleges and universities currently undertake 
periodic alumn! or former student surveys, but they rarely use them 
to their fullest potential. Most such surveys are limited to 
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postgraduate job and graduate school placement. Adding to such 
efforts by Including Items on personal and educational goal 
achievement, om self-reported gains In Identified knowledge and skill 
areas, and on a range of current behaviors and values can provide a 
rich source of Information useful for both external accountability 
and to Inform local Improvement, Finally, Information of this kind 
Is not directly threatening to faculty; far more faculty cooperation 
and Interest can be expected In this arena than in cognitive testing 
where the stakes to faculty appear much higher. 

Beware the "Unaccompanied Number". Certainly the greatest single 
Institutional fear with respect to externally mandated assessment is 
that overall Instructional effectiveness will be judged on the basis 
of a few standardized statistics. Most emerging state programs do 
not seem to be emphasizing this approach, but prudent Institutions 
must still anticipate misuse of publicly communicated assessment 
results. How Is this best accomplished! A general answer Is to 
ensure that supplementary Information always be provided, to place 
all required outcomes Information fn Its proper context* 
Specifically, two kinds of contextual Information need to be 
provided. The first Is Information that allows an audience to judge 
the reliability and Importance of a given result* If follow-up 
surveys, for example. Indicate that students are experiencing 
difficulty In obtaining the entry-level Jobs for which they have been 
trained, additional Information on the depressed state of key sectors 
of the local economy may help to explain the finding. If results of 
student performance on standardized examinations Is required, these 
results might be accompanied by additional Information that confirms 
or disconf Irms the reported result — for example, student performance 
on other examinations, or by Information that helps to explain the 
result — for example, performance on entering aptitude or placement 
examinations, evidence of deficient high school course-taking, and so 
on, A second kind of context Is provided by additional Information 
on Intended action to address Identified deficiencies. Communicating 
Institutional plans together with required assessment results 
demonstrates that the Institution takes the assessment process 
seriously and intends to do something about It. Certainly In neither 
case Is there a guarantee that providing additional Information will 
prevent the abuse of performance statistics. But It may forestall 
more obtrusive action. At the very least. If public criticism 
arises, the Institution wil I already be on record with a response. 

Offer information on student outcome s to external constituencies even 
if unasked. This strategy emphasizes the Importance of taking every 
opportunity to share Important results of local assessment. As noted 
throughout. It Is critical that Institutions demonstrate that they 
are not In principle opposed to assessment, and that they In fact 
engage In a lot of It, Existing external reporting requirements 
provide many opportunities for an Institution to actively demonstrate 
that It has made a commitment to using assessment results. Among 
them are such state-mandated processes as program approval and 
program review, mission review and strategic planning processes, and 
accreditation self-stuoy reports. Though outcomes Information may 
not be required In such processes or may play only a minor role. 



including it on the Institution's own initiative highlights its 
assessment capabilities, and demonstrates its knowledge of how such 
information can be used to improve practice. Indeed, external 
constituents v?ill Mkely judge the adequacy of a local assessment 
effort partly by its consistency: how often and in how many 
different areas does the institution demonstrate that it is taking 
this information seriously? Given this unspoken criterion, 
institutions should take every available opportunity to tell their 
own stories with respect to outcomes* 

Make "assessment" a visible part of the rurr feu I um . Senior 
comprehensive examinations and the use of external examiners 
represent special cases of a more general institutional 
response — emphasizing the presence of assessment throughout the 
curriculum. Student perforiiance In most curricula Is evaluated 
constantly, but the processes used are both Ill-documented and are 
unevenly practiced. How can these deficiencies be addressed? One 
promising approach Is simply to document the curriculum more fully. 
To communicate effectively, curriculum design should emphasize Its 
Intended outcomes as explicitly as possible. Moreover, It should 
note the kinds of generic skills that students are intended to gain, 
as well as the disciplinary knowledge that the curriculum 
encompasses. Once provided, curriculum goals of this kind provide an 
Important starting place for local assessment. But their presence 
and visibility In public documents sends an additional Important 
message to external constituents. At the same time, the institution 
should highlight developing faculty efforts to assess student 
performance. Descriptions of senior comprehensive examinations, 
projects, or "capstone experiences" should contain some indications 
of how student performance will be assessed — particularly if the 
process Involves team evaluation or the use of practicing 
professionals. The same principles ultimately apply to the 
documentation of each course. Its requirements, and its examinations. 

MflKe asserr.ment results a vi<;Ible part o f important ;^dm in; strati ve 
processes, a parallel response involves documenting the role of 
assessment results in important^ regular administrative processes. 
Examples Include course approval, program review, strategic planning, 
and Institutional budgeting. In contrast to documenting existing 
assessment practice In the curriculum, here the use of Information on 
instructional outcomes will in most cases be novel. As yet, the 
majority of Institutions have not accorded a prominent place for 
outcomes information In ongoing administrative decisionmaking. Here 
again, assessment results are best used in conjunction with nKire 
familiar management information, as an integral part of established 
decision mechanisms. Such processes themselves, however, are often 
badly documented. Building externel credibility for institutional 
assessment practice may require making visible such linkages and 
showing that they make a difference. Documents such as local 
planning manuals, program review guidelines, budget reviev- and 
preparation guidelines, and institutional "Fact Books" should all 
accord a visible place for information on student outcomes, and a 
review of the actual decisions made should reflect full consideration 
of such information. 
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- Cctiimuplcate assessment practices, not )us t results ^ Asked or 
unasked, 'v !s as Important for Institutions to communicate their 
processes of assessment as It Is to present the outcomes themselves* 
External authorities need to be able to present evidence of action to 
their own constituencies — legislators and the wider public* To the 
degree that they have reassuring stories to tell. Individual 
institutions and higher education In general will likely be allowed 
greater discretion* If little Information Is reported, external 
authorities may assume the worst: that the institution is not 
intending to comply* Because comprehensive institutional assessment 
programs will require some time to get established (three years Is a 
corritton experience for those currently engaging In such efforts), for 
a while there wll I be few actual results to communicate* This 
Inevitable lag calls particular attention to the need to provide 
Information about the assessment process* Several states now require 
Institutions to submit locally developed assessment plans documenting 
what they ?r.tend to do* Even where such plans are not explicitly 
required, th^Ir development and dissemination to appropriate external 
constItuenci3S appears a good Idea* Where such plans are required. 
Institutions should on their own Initiative provide periodic 
additional Information on how they are being Implemented and 
modified. Once again, existing reporting opportunirles such as 
statewide program reviews, strategic planning exercises, or 
Institutional self-studies should be fully utilized to communicate 
assessment plans and practices, 

" Don^t ghow everything at once. Tempering the above observation, 
external authorities also value novelty* They particularly want to 
be able to cite new institutional practices, consistent with the 
guidelines that they provide* Furthermore, novelty provides evidence 
of ongoing commitment* If institutions appear to undertake new 
Initiatives each year, they also show that they remain seriously 
committed to assessment. Too much novelty, of course, can send a 
negative message; certainly any "new' Initiatives reported must be 
consistent both with one another and with the Institution's past 
practices with regard to assessment* But It Is probably wise for 
institutions to keep a few of their asirissment pracrlces and results 
In reserve, both to have something new each year to promise, and to 
have something new to report* 

Through a I I these suggested strategies runs a common theme: in assessment as 
In other areas, accountability Is a bargain between two parties. As such, it 
Is subject to specified terms and conditions, and depends upon the mutual good 
faith with which these are observed by the respective parties. In keeping 
their part of the bargain. Institutions must recognize the legitimacy of many 
external requests for outcomes Information, It Is both inappropriate and 
ultimately ineffective for them to refuse cooperation In providing the 
Information that external constituencies need to discharge their assigned 
responsibilities. At the same tlm^^ It Is Incumbent upon institutions to 
continually remind external authorities of tlieir part of the bargain. Requests 
for Information about student instructional performance must be carefully 
limited to areas of legitimate external concern. Outside these areas. 
Institutions ought to be allowed, and shouM Insist upon, maximum discretion. 
In exercising discretion, however. Institutions shojid also keep external 
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constituencies fully informed about the processes they have established and the 
actions thoy have taken. Above all. It is Important for each institution to 
demonstrate good faith in Its efforts to comply with the spirit of external 
assessment demands. Cooperative efforts will in the long run prove less 
painful than the appearance of non-compliance. More importantly, demonstrating 
cooperation can further local Interest. Compliance, in itself, is a barren 
exercise. Taking the initiative to move beyond compliance is an institutional 
response that will allow the legitimate Interests of both parties to prosper. 
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